fix(ci): improve auto bm note narrative#893
Conversation
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 2m 0s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
Code Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
FindingsMinor:
|
|
Claude finished @phernandez's task in 2m 11s —— View job PR Review —
|
| Item | Status |
|---|---|
| Python 3.12+ type annotations | ✅ |
| Typer conventions followed | ✅ |
| 100-char line length | ✅ |
| No commented-out code | ✅ |
| No hardcoded secrets | ✅ (env vars only) |
| No emojis/unicode in console output | ✅ |
| Windows file path compat | ✅ (as_posix() used in _note_write_target) |
Observations
1. API failure propagation is intentional but worth confirming
_enrich_pull_request_context in project_updates.py:371 raises ValueError on any GitHub API error (HTTP error, network error). This propagates to collect_project_update_context → CLI collect → exit code 1, failing the CI step entirely.
The README explicitly documents this: "If the token is present and GitHub API enrichment fails, the Auto BM workflow fails fast instead of publishing a weak note."
That's a deliberate design call and matches the project's fail-fast philosophy. No action needed — just confirming this is intentional.
2. Missing test: API error path for enrichment
There's a test for the happy-path enrichment (test_collect_enriches_pull_request_context_from_github_api at tests/ci/test_project_updates.py:98) but no test for when GITHUB_TOKEN is present and the API call returns a 4xx/5xx. Given the README documents this behavior explicitly, a test like test_collect_fails_when_github_api_returns_error would complete the coverage contract for the new error path.
3. _github_api_get does not handle JSON decode errors from the API body
project_updates.py:285-286:
with urllib.request.urlopen(request, timeout=20) as response:
payload = json.loads(response.read().decode("utf-8"))If GitHub returns a non-JSON body (e.g. a 200 with a maintenance HTML page), this raises an undecorated json.JSONDecodeError, not a ValueError with context. The collect CLI command only catches ValueError, so this would surface as an unhandled exception traceback rather than a clean error message.
Low probability in practice (GitHub API is reliable), but a try/except json.JSONDecodeError wrapping with a raise ValueError(...) would be consistent with the rest of the module's error handling style.
4. changed_files_count and changed_files can diverge
ProjectUpdateContext has both changed_files_count (from the PR payload integer field) and changed_files (list populated by API enrichment). After enrichment, len(changed_files) may differ from changed_files_count if the GitHub API paginates or truncates (GitHub caps files at 3000 in the REST API). Not a bug — these are distinct concepts (payload count vs fetched detail list) — but a future reader might not notice the difference. The note renderer only uses the list, so no behavioral issue here.
5. AgentSynthesis.model_config = ConfigDict(extra="ignore") with required fields
project_updates.py:110: extra="ignore" is retained from before, which means the model silently drops unknown fields from Codex output. The new required fields (story, problem_addressed, solution, system_impact) are all validated via _required_text to be non-empty strings. If Codex returns an extra field (e.g. legacy title or intent from an older schema), it's dropped cleanly. ✅
Testing & Documentation
| Item | Status |
|---|---|
| Unit tests for new API enrichment | ✅ (test_collect_enriches_pull_request_context_from_github_api) |
| Tests for required narrative fields | ✅ (test_agent_synthesis_requires_delivery_narrative_fields, test_build_project_update_note_renders_story_sections) |
## Project Memory / no ## Why It Matters test |
✅ (test_build_project_update_note_renders_story_sections:419-420) |
| API error path test | |
| README updated | ✅ |
memory-ci-capture.md updated and matches render_capture_prompt() |
✅ (checked-in file matches rendered output) |
Checklist
- Follows Basic Memory coding conventions
- Python 3.12+ type annotations and async patterns
- N/A — no SQLAlchemy changes
- FastAPI and Typer conventions followed
- 100-character line length maintained
- No commented-out code blocks
- Unit tests for new functions/methods
- N/A — no new MCP tools
- [~] Test coverage for edge cases — API error path missing
- [~] 100% test coverage — unverified for new
json.JSONDecodeErrorpath in_github_api_get - Documentation updated
- N/A — CLAUDE.md conventions unchanged
- N/A — no new MCP tools
- N/A — no database changes
- Preserves local-first architecture principles
- N/A — no knowledge graph changes
- N/A — no markdown file handling changes
- No hardcoded secrets or credentials
- Input validation for new models
- Proper error handling (with one minor gap noted)
- Performance considerations addressed (pagination handles >100 files)
- No sensitive data in logs
- Windows-compatible file path comparisons
- No emojis or unicode in console/log output
Two minor items to consider: add a test for the API failure path (observation #2) and optionally harden the JSON decode path in _github_api_get (observation #3). Neither blocks merge.
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 2m 48s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
SummaryThis PR enriches Auto BM's PR context with changed files, commits, and linked issue details from the GitHub API; makes the Team ChecklistCode Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
Issues1. Test isolation:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d47c8c1647
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| project=project, | ||
| project_id=project_id, | ||
| workspace=workspace, | ||
| refresh=refresh_schemas, |
There was a problem hiding this comment.
Let schema refresh run in installed repos
When --refresh-schemas is used in a repo that already has Auto BM installed, execution never reaches this refreshed seed call because _write_generated_files(...) above still rejects the existing workflow/config/prompt unless --force is also passed. That makes the new refresh/update flags unusable for their documented upgrade path of refreshing schema notes without overwriting generated files; skip or relax the generated-file write when only refreshing schemas.
Useful? React with 👍 / 👎.
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 2m 25s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
SummaryWell-scoped PR that enriches the Auto BM context with changed files, commits, and linked issue details from the GitHub API; makes narrative fields required on Team ChecklistCode Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
Issues1. Test isolation:
|
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 2m 25s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
SummarySolid, well-scoped PR. The GitHub API enrichment layer (changed files, commits, linked issue details), required narrative fields on Team ChecklistCode Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
Issues1. Test isolation:
|
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 1m 57s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
SummarySolid enrichment of the Auto BM CI layer. The GitHub API enrichment (changed files, commits, linked issue details), required narrative fields on Team ChecklistCode Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
Issues1. Test isolation:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bbf32b4ec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| detail = _issue_detail_from_github( | ||
| _github_api_get(f"/repos/{context.repo}/issues/{number}", token) |
There was a problem hiding this comment.
Treat missing linked issue details as best-effort
Because the generated workflow now always sets GITHUB_TOKEN, every eligible PR body/title containing a #number reaches this lookup. If that number is not an issue in the same repo (for example text like attempt #2, an RFC number, or a deleted issue), GitHub returns 404 and _github_api_get raises ValueError, causing bm ci collect to fail before any project update is published. The issue-detail enrichment should skip unavailable issue details rather than making the whole collection step fail.
Useful? React with 👍 / 👎.
| await mcp_write_note( | ||
| title=spec.title, | ||
| title=title, | ||
| content=spec.content, | ||
| directory="schemas", | ||
| directory=directory, | ||
| project=routed_project, |
There was a problem hiding this comment.
Preserve the schema file name when refreshing
When --refresh-schemas finds an existing schema note whose filename does not match its title, this call still passes only the existing title and parent directory to write_note. Since write_note derives the target path from title + directory, overwrite=True will create/update custom/schemas/<title>.md instead of the matched file_path (for example custom/schemas/update.md), so refresh can duplicate the schema note rather than updating the installed one. Use the matched path/permalink, or otherwise derive the write target from the existing filename, when refreshing.
Useful? React with 👍 / 👎.
Signed-off-by: phernandez <paul@basicmachines.co>
|
Claude finished @phernandez's task in 2m 12s —— View job PR Review: fix(ci): improve auto bm note narrativeTasks:
SummaryWell-scoped PR. The GitHub API enrichment (changed files, commits, linked issue details), required narrative fields on Team ChecklistCode Quality & Standards
Testing & Documentation
Basic Memory Architecture
Security & Performance
Compatibility
Issues1. Functional: linked issue 404 can silently fail the CI collect step
This is a genuine production breakage risk: a merged PR with a body containing # Current — fails the whole step if any issue is 404
detail = _issue_detail_from_github(
_github_api_get(f"/repos/{context.repo}/issues/{number}", token)
)
# Better — skip issues that can't be fetched
try:
raw = _github_api_get(f"/repos/{context.repo}/issues/{number}", token)
detail = _issue_detail_from_github(raw)
except ValueError:
detail = None2. Test isolation:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b828aaf055
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| files = { | ||
| repo_root / DEFAULT_WORKFLOW_PATH: render_workflow(config), | ||
| repo_root / DEFAULT_PROMPT_PATH: render_capture_prompt(), | ||
| repo_root / DEFAULT_SOUL_PATH: render_soul_template(), |
There was a problem hiding this comment.
Seed the new soul guide without forcing overwrites
When upgrading a repo that already has the previous Auto BM workflow/config/prompt but does not yet have SOUL.md, adding the soul guide to this all-or-nothing generated-file set leaves no non-destructive setup path: normal setup aborts on the existing files, --refresh-schemas returns before writing the missing soul file, and --force rewrites any customized workflow/config/prompt. Please treat SOUL.md as a missing-only seed, or write missing generated files before rejecting existing ones.
Useful? React with 👍 / 👎.
Summary
Verification